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, Abstract. This p aper analyzes the performance of observatories based on 

the so-called /i- index | ijHirsch 2005j) . a new, easy-to-use parameter that quanti- 
fies scientists' research impact and relevance. Compared to other bibliometric 
criteria, like total number of publications or citations, the /i-index is less biased. 
J> ' Using NASA's Astrophysics Data System (ADS), we investigate the performance 

\ of selected observatories, taking into account their specific number of years of 

■ operation. 
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1. Telescope Statistics 



At many observatories, librarians are engaged in compiling telescope statistics. 
These compilations of papers that use data coming from specific observing fa- 
53 . cilities are needed for a number of purposes, among them assessing the overall 

impact of facilities, measuring the scientific output from observing programs, 
and reporting back to funding authorities. Despite the intrinsic dangers (see 
rS \ below), telescope statistics are also often used to compare output among obser- 

vatories. 

Certain bibliometric methods are used frequently in order to evaluate fa- 
cilities. These methods usually show some advantages as well as specific dis- 
advantages. For instance, counting the total number of publications measures 
productivity of an observatory, but does not take the importance and impact 
into account which the papers resulting from the data have generated. In con- 
trast, looking at the total number of citations does measure impact, but the 
value may be inflated by a few papers that received extraordinarily high num- 
bers of citations. Another method is to investigate contributions to the so-called 
High- Impact Papers, a term originally coined by Thomson ISI for their database 
of the most influential papers in specific fields. In astronomy, this name is of- 
ten used to refer to a study performed by the ST Scl, where the 200 highest 
cited observational papers are distributed across observatories, based on their 
actual input of data to these papers (see Madrid, these proceedings). Measuring 
contributions to High-Impact Papers is more balanced than other bibliometric 
methods, but the values per observatory are much more difficult to find. 
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2. The /i-Index 



In 2005, Jorge E. Hirsch of the Universi ty of Cahforn ia at San Diego introduced 
a new measure, the so-called /i-index I (|Hirsch 200,ll V The original aim was to 
quantify an individual's scientific research output. According to the definition 
by Hirsch, a researcher with index h has h papers with at least h citations. In 
other words, index h is "the highest number of papers a scientist has that have 
each received at least that number of citations" (Ball 2005). For instance, if a 
researcher has written 50 papers, 30 of which have achieved 30 or more citations, 
his or her /i-index is 30. 

In order to find the /i-index, one needs a list of all papers fulfilling the 
criteria under investigation (e.g., written by a specific person or using data from 
a given facility). This list must be numbered (from 1 to the total number of 
papers in the list) and ranked by decreasing citation counts, i.e., the paper with 
the highest number of citations appears on top, that with the lowest number at 
the bottom, h can be found where the number of citations are at least as high 
as the rank (publication number). See Fig. 1 for an example, using the ADS. 
Fig. 2 demonstrates that neither a large number of papers with few or no cites 
(x-axis, right end) nor individual papers with a very high number of citations 
(y-axis, upper end) influence h considerably. 
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Figure 1. How to find the /i- index with the ADS: the highest paper number 
that has at least as many citations. 
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Figure 2. /i- index: where the rank (counted pubhcation number) and the 
number of citations are the same. 



3. Applying h to Observatories 



The authors assumed that what works weh for individual researchers should 
also be applicable to facilities, and applied Hirsch's h-'mdex to observatories. 
Our study includes papers from the CFHT, HST, Keck, ESO VLT and Gemini. 
This selection is somewhat arbitrary as it was largely defined by the question 
of which observatories provide easy access to bibcodes of papers resulting from 
their data. 

Before we continue, we would like to issue a word of warning: it is very 
difficult, if not impossible, to compare statistics across observatories in a fair, 
unbiased way. Some of the reasons originate from the procedures - the crite- 
ria for inclusion of papers are not uniform so that the contents of the resulting 
bibliographies vary. Also the methodologies applied for building telescope bib- 
liographies vary among institutions; they can range from electronic searching 
whatever sections of relevant journals are available online to scrutinizing jour- 
nal issues on paper (with the non-exclusively electronic approach still leading to 
a higher completeness, see Stevens-Rayburn & Grothkopf, these proceedings). 
Other difficulties result from the differences among facilities: we only have to 
think of ground-based versus space-based observatories, different apertures, dif- 
ferent wavelengths etc. Therefore, our study is meant as a first introduction 
of the h-'mdex in the evaluation of facilities; further studies will be needed to 
fine-tune the methodology. 
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4. Our Sample 

As mentioned above, our sample includes those telescope bibliographies to which 
we had comparably easy access. These are the following observatories and ranges 
of years of coverage: 

Table 1. Total sample included in our study. 

CFHT 1980 - 2005 

HST 1991 - 2005 

Keck 1996 - 2005 

ESO VLT 1999 - 2005 

Gemini 2000 - 2005 



For uniformity, we ended the sample with publication year 2005 for all 
facilities. 

For the years given above, we collected all bibcodes of papers included 
in the observatories' telescope bibliographies as follows: for CFHT and HST, 
bibcodes were retrieved through the ADS by using the filter "Select Refer- 
ences In:" CFHT and HST, respectively, sorted by citation count. (This fil- 
ter also allows to select ESO Telescopes; however, this feature does not dis- 
criminate between ESO VLT and ESO La Silla papers. Such a distinction is 
only possible through the ESO Telescope Bibliography, see below.) Keck bib- 
codes were extracted from the references provided in their Science Bibliogra- 
phy (http://www2.keck.hawaii.edu/library/1996.htm through 2005.htm). Gem- 
ini bibcodes and numbers of citations were kindly provided by the Gemini li- 
brarian. Bibcodes of papers using ESO VLT data are available through the ESO 
Telescope Bibliography (http:/ /www.eso.org/libraries/telbib.html). 

Individual citations per paper were retrieved from the ADS and (in the case 
of Keck, VLT and Gemini) listed in decreasing order in an Excel spreadsheet 
so that the /i-index could be obtained. For CFHT and HST, the ADS very 
conveniently provides the /i-index as described above. 

Computing the h-index for all years included in our study led to the fol- 
lowing results: CFHT (1980 - 2005): 94; HST (1991 - 2005): 131; Keck (1996 - 
2005): 105; ESO VLT (1999 - 2005): 69; Gemini (2000 - 2005): 26. Note that 
in order to be meaningful, h needs to be complemented with a parameter that 
normalizes based on the number of years of operation! 

5. /i-Index and m Parameter 

Obviously, the telescopes we investigated did not come online at the same time, 
but over a period of 20 years (see Table 1). Accordingly, some had more, others 
much less time to gather citations. We therefore applied a measure to normalize 
results based on the time that has lapsed since the first papers were published: 
the m parameter, also introduced by Hirsch (2005). In simple words, m is h 
divided by the number of years of operation. For instance, if h is 20 after 20 
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years, m is 1 (20 : 20 = 1); if /i is 40 after 20 years, m is 2 (40 : 20 = 2) etc. 
The higher m is, the better. 

In order to retrieve m easily for our sample, we introduced a "Years since 
first paper" column in our table and divided h by the number of years indicated 
in this column. The results are shown in Table 2 and Fig. 3. 

Table 2. /i-index and parameter m computed for all publication years in- 
cluded in our study. 



Observatory Years Years since first paper h m 

CFHT 1980 - 2005 25 94 3.76 

HST 1991 - 2005 14 131 9.36 

Keck 1996 - 2005 9 105 11.67 

ESO VLT 1999 - 2005 6 69 11.5 

Gemini 2000 - 2005 5 26 5.2 





h index parameter m 



Figure 3. h-'mdex and parameter m of our entire sample (see Table 1). Note 
that h needs to be complemented with m to be meaningful. 



6. h and m for Identical Range of Years 

In order to focus on common years of operation, we went on to investigate those 
telescopes for which we had bibliographies stemming from the same period, 
namely CFHT, HST and Keck. For these facilities, we limited the range of years 
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to 1996 - 2005 and again computed h and m. The results are given in Table 3. 
While h and m were vastly different when looking at all years, the values take 
on a much more similar shape for the subset of years (even though on a different 
scale, see Fig. 4), with HST being the leader, followed by Keck and CFHT. 
In comparison to results of our entire study (from the first papers onwards), 
m- values of the selected years 1996 to 2005 have increased for the two older 
telescopes (CFHT and HST), suggesting that the metric doesn't flow as linearly 
as one might desire, but rather new instrumentation or more creative observing 
proposals can positively affect the citation rates for older observatories. 

Table 3. h-'mdex and parameter m computed for the years 1996 - 2005. 



Observatory 


Years 


Years since first paper 


h 


m 


CFHT 


1996 - 


2005 


9 


62 


6.89 


HST 


1996 - 


2005 


9 


118 


13.11 


Keck 


1996 - 


2005 


9 


105 


11.67 




Figure 4. /i-index and parameter m of selected telescope bibliographies for 
the years 1996 - 2005. 

For the HST, we conducted another brief study in order to find out if and 
how h and m change over time. For all HST papers, going back to the first ones 
published after the launch in 1991, and including the first six months of 2006, 
we found h = 132 and m = 8.8. If the early papers are left out and the range 
of years taken into account starts after the first servicing mission (COSTAR) in 
1994, these values change to h = 128 and m = 10.66. STIS and NICMOS were 
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installed in 1998, and h and m for the years 1998 to mid-2006 are 103 and 12.87, 
respectively. This indicates that also the m parameter can be misleading with 
regard to telescopes that continue to be refurbished over time. Both h and m 
therefore should be used in telescope statistics with greatest care. 

7. Conclusions 

There are some clear advantages of the /i-index which may make it suitable also 
for telescope statistics: 

• it combines productivity and impact 

• it is easy to find using the ADS 

• it is not sensitive to extreme values 

On the other hand, as with all statistics, one has to apply utmost attention 
when applying the /t-index, in particular because 

• it is determined by the number of years of operation and 

• it needs to be combined with the m parameter for comparison across fa- 
cilities 

In this paper, we have shown how the /t-index and the m parameter can 
be used in telescope statistics. However, it is important to note that one single 
parameter is never enough to represent the complex reality, and a narrowly based 
approach might lead to incorrect results and wrong interpretations. 
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